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DETAILED ACTION 

This Office Action corresponds to application 10/815,074. 



Reopening of Prosecution after Appeal Brief 

In view of the Pre-Appeal Brief request filed on 2/12/2009, PROSECUTION IS 
HEREBY REOPENED. A new ground of rejection is set forth below. 

To avoid abandonment of the application, appellant must exercise one of the 
following two options: 

(1) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply 
under 37 CFR 1.113 (if this Office action is final); or, 

(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41 .31 followed 
by an appeal brief under 37 CFR 41 .37. The previously paid notice of appeal fee and 
appeal brief fee can be applied to the new appeal. If, however, the appeal fees set forth 
in 37 CFR 41.20 have been increased since they were previously paid, then appellant 
must pay the difference between the increased fees and the amount previously paid. 

A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by 
signing below: 



/John R. Cottingham/ 

Supervisory Patent Examiner, Art Unit 2167 
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Claim Objections 

Claims 1 and 18 are objected to because tliey sliould include a semicolon (;) 
after "result" in the determining step. 

Appropriate correction is required. 

35 use §101 

Claims 18-21 are now accepted under 35 U.S.C. 101 for the inclusion of a 
computer-readable storage medium. As an interpretation gleaned from the specification 
(e.g. paragraph 0012), a computer-readable storage medium is best interpreted as 
electronic, optical, or magnetic media (such as a hard drive) and thus can be seen as 
statutory. Because these claims preclude the use of transmission media, the claims are 
seen as statutory. 

Claims 25-28 are now accepted under 35 U.S.C. 101 for the inclusion of a 
computer processor which is best seen as a hardware element, thus precluding the 
interpretation of claiming software per se. 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 
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(e) the invention was described in (1) an application for patent, publislied under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

Claims 1-2, 4-12, 15-18, and 21-26 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Liu at al. ('Liu' hereafter) (U.S. Patent 7,099,860). Liu teaches the claims 
in the following drawing references of figures 1-7 and the following cited portions. 

With respect to claim 1, Liu teaches A computer-implemented method for ranking 
information, comprising: 

receiving a plurality of query results (col. 3 line 32-34 and figures 6-7; e.g. 
retrieving a plurality of images in response to a search) of a plurality of search queries 
(col. 3 line 23-25 and col. 7 line 67-col. 8 line 10 as well as figure 3 drawing reference 
302 and col. 12 step 1 ; e.g. keywords parsed from a query); 

merging (col. 8 line 22-23; i.e. ac/c//ngf found images to the result set) the plurality 
of query results (col. 3 line 32-34 and figures 6-7) into a merged query result (figs. 6-7), 
the merged query result (figs. 6-7) being associated with the plurality of search queries 
(col. line 20-24 and figure 3 which shows keywords associated to images); 

determining a first ranking sequence (fig. 4 drawing reference 412) of the merged 
query result (col. 7 line 15-18figs. 6-7); 
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presenting tlie merged query result (figs. 6-7) to a user (figure 2 drawing 
reference 200) according to the first ranking sequence (col. 7 line 19-21 and drawing 
reference 416; i.e. displaying a result set from initial query handling); 

identifying an input signal (figure 5 drawing reference 502) from the user (200) 
indicating an interest (col. 7 line 22-23) in a first piece of information (col. 7 line 22-23; 
i.e. an image a user finds relevant/not relevant) in the merged query result (figures 6-7); 

identifying a search query from the plurality of search queries (col. 3 lines 41-48, 
col. 10 step 4) associated with the merged query result (figs. 6-7), the identified search 
query (col. 3 lines 41-48, col. 10 step 4) being associated with a query result (figure 3 
drawing reference 304 and col. 8 line 57; e.g. an association link) including the first 
piece of information (col. 6 line 8-19), the query result from among the plurality of query 
results (col. 10 step 4; "for each positive example, check if any query is linked to it..." ; 

adjusting (col. 3 line 49-50) a query factor associated with the identified search 
query (col. 5 line 24-29; e.g. a weight indicating a relevance of a keyword to an image) 
responsive to the input signal (col. 8 line 52-64); 

locating a second piece of information in the query result (col. 9 line 13-19;"the 
feature and semantic matcher 152 also try to locate images that have similar low-level 
features as the example image selected by the user"; herein, the semantic and low-level 
features can be interpreted as an example of the "second piece of information in the 
query result") associated with the identified search query (col. 3 lines 41-48, col. 10 step 
4); 
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determining a score for tine second piece of information (col. 9 line 16-19) based 
at least in part on the query factor col. 5 line 24-29; e.g. a weight indicating a relevance 
of a keyword to an image) associated with the identified search query (col. 3 lines 41- 
48, col. 10 step 4); 

determining a second ranking sequence (col. 12 step 7) of the merged query 
result based at least in part on the score (col. 7 line 49-50 and figure 5); and 

presenting the merged query result to the user according to the second ranking 
sequence (col. 12 step 8 and drawing references 508m, 518 as well as figures 6-7). 

With respect to claim 2, Liu teaches the method of claim 1, wherein the input 
signal indicates a selection of the first piece of information (figure 5 drawing reference 
504). 

With respect to claim 4, Liu teaches the method of claim 1, wherein the input 
signal comprises user activity associated with the first piece of information (col. 8 line 
45-51). 

With respect to claim 5, Liu teaches the method of claim 4, wherein the user 
activity comprises one or more of viewing duration, scrolling, mouse movement, 
selection of links from the first piece of information, saving, printing, and bookmarking 
(col. 8 line 47-48 describes at least "mouse movement" as well as "bookmarking"). 
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With respect to claim 6, Liu teaches the method of claim 4, wherein the input 
signal further comprises user activity associated with articles linked from the first piece 
of information (col. 9 lines 61-67). 

With respect to claim 7, Liu teaches the method of claim 1, further comprising: 
identifying parts of text typed by the user (col. 5 line 67-col. 6 line 3; "a natural 
language parser 202 to parse text-based queries, such as keywords phrases and 
sentences"), the parts including at least two of the following: nouns, verbs, and proper 
nouns (col. 8 lines 1-10 wherein Liu describes extracting keywords "tigers", "pictures", 
and "images" to teach identifying at least two nouns as well as separating these 
keywords from "looking for" and "find" which can be seen as identified verbs); and 

generating the plurality of search queries based on the identified parts (col. 8 line 
5-10 and drawing reference 408). 

With respect to claim 8, Liu teaches the method of claim 1 , wherein the input 
signal comprises a user rating (col. 9 lines 64-67). 

With respect to claim 9, Liu teaches the method of claim 1 , wherein one of the 
plurality of search queries comprises one of query type, query term, application, type of 
application, article type, and event type (col. 5 line 16 and col. 6 line 1). 
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With respect to claim 10, Liu teaches the method of claim 9, wherein the query 
type comprises one of current sentence, current paragraph, text near the cursor, 
extracted terms, and identified entries (col. 6 line 2; e.g. extracted keywords). 

With respect to claim 11, Liu teaches the method of claim 1, wherein the score 
comprises a relevance score (col. 5 line 30-31). 

With respect to claim 12, Liu teaches the method of claim 1, wherein the score 
comprises a popularity score (col. 10 Iine21; e.g. a voting scheme describes 

"popularity"). 

With respect to claim 15, Liu teaches the method of claim 1, wherein the input 
signal comprises multiple input signals (col. 9 line 52-67; e.g. "view", rating and "similar" 
are all inputs effected to suggest a users interest). 

With respect to claim 16, Liu teaches the method of claim 1, further comprising: 
generating the plurality of search queries based on a plurality of data streams 
(col. 8 line 1-4); 

and executing the plurality of search queries for the plurality of search results 
(figure 4, drawing reference 406). 
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With respect to claim 17, Liu teaches the method of claim 16, wherein the 
plurality of data streams comprise a data stream describing current contextual state of a 
user (figure 5 drawing reference 504-518; e.g. finding images based on a selected 
image describes implicitly finding related images to a user's context (e.g. preference). 

With respect to claim 18, Liu teaches A computer program product having a 
computer-readable storage medium having executable computer program instructions 
tangibly embodied thereon for ranking information, the executable computer program 
instructions comprising instructions for: 

receiving a plurality of query results (col. 3 line 32-34 and figures 6-7; e.g. 
retrieving a plurality of images in response to a search) of a plurality of search queries 
(col. 3 line 23-25 and col. 7 line 67-col. 8 line 10 as well as figure 3 drawing reference 
302 and col. 12 step 1 ; e.g. keywords parsed from a query); 

merging (col. 8 line 22-23; i.e. adding iound images to the result set) the plurality 
of query results (col. 3 line 32-34 and figures 6-7) into a merged query result (figs. 6-7), 
the merged query result (figs. 6-7) being associated with the plurality of search queries 
(col. line 20-24 and figure 3 which shows keywords associated to images); 

determining a first ranking sequence (fig. 4 drawing reference 412) of the merged 
query result (col. 7 line 15-18figs. 6-7); 

presenting the merged query result (figs. 6-7) to a user (figure 2 drawing 
reference 200) according to the first ranking sequence (col. 7 line 19-21 and drawing 
reference 416; i.e. displaying a result set from initial query handling); 
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identifying an input signal (figure 5 drawing reference 502) from the user (200) 
indicating an interest (col. 7 line 22-23) in a first piece of information (col. 7 line 22-23; 
i.e. an image a user finds relevant/not relevant) in the merged query result (figures 6-7); 

identifying a search query from the plurality of search queries (col. 3 lines 41-48, 
col. 10 step 4) associated with the merged query result (figs. 6-7), the Identified search 
query (col. 3 lines 41-48, col. 10 step 4) being associated with a query result (figure 3 
drawing reference 304 and col. 8 line 57; e.g. an association link) including the first 
piece of information (col. 6 line 8-19), the query result from among the plurality of query 
results (col. 10 step 4; "for each positive example, check if any query Is linked to It..." ; 

adjusting (col. 3 line 49-50) a query factor associated with the Identified search 
query (col. 5 line 24-29; e.g. a weight indicating a relevance of a keyword to an image) 
responsive to the input signal (col. 8 line 52-64); 

locating a second piece of Information in the query result (col. 9 line 13-19; "the 
feature and semantic matcher 152 also try to locate images that have similar low-level 
features as the example image selected by the user"; herein, the semantic and low-level 
features can be interpreted as the "second piece of information") associated with the 
identified search query (col. 3 lines 41-48, col. 10 step 4); 

determining a score for the second piece of information (col. 9 line 16-19) based 
at least In part on the query factor col. 5 line 24-29; e.g. a weight indicating a relevance 
of a keyword to an image) associated with the identified search query (col. 3 lines 41- 
48, col. 10 step 4); 
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determining a second ranl<ing sequence (col. 12 step 7) of the merged query 
result based at least in part on the score (col. 7 line 49-50 and figure 5); and 

presenting the merged query result to the user according to the second ranking 
sequence (col. 12 step 8 and drawing references 508m, 518 as well as figures 6-7). 

With respect to claim 21, Liu teaches the computer program product of claim 18, 
the executable computer program instructions further comprising instructions for: 

generating the plurality of search queries based on a plurality of data streams 
(col. 8 line 1-4); 

and executing the plurality of search queries for the plurality of search results 
(figure 4, drawing reference 406). 

With respect to claim 22, Liu teaches the method of claim 1, wherein determining 
the second ranking sequence comprises: 

determining the second ranking sequence of at least some of the merged query 
result based at least in part on the score (figure 5 and col. 12 steps 7-8), the at least 
some of the merged query result associated with at least two search queries (figure 3 
and col. 12 step 1). 

With respect to claim 23, Liu teaches the method of claim 1 , further comprising: 

generating the plurality of search queries (col. 8 line 1-10); and 

adding information from results of the plurality of search queries into the merged 
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query result (col. 8 line 23-25 and figure 4). 

With respect to claim 24, Liu teaches the computer program product of claim 18, 
the executable computer program instructions further comprising instructions for: 

generating the plurality of search queries associated with the merged query 
result (col. 6 line 38-47); 

and adding information from results of the plurality of search queries into the 
merged query result (col. 8 line 23-25 and figure 4). 

With respect to claim 25, Liu teaches A query system for ranking information, 
comprising: 

a computer processor (130) for executing computer program instructions (col. 7 
line 52); 

a computer-readable storage medium having executable computer program 
instructions (132, 134) tangibly embodied thereon, the executable computer program 
instructions comprising instructions for: 

a module (150) configured to receive a plurality of query results (col. 3 line 32-34 
and figures 6-7; e.g. retrieving a plurality of images in response to a search) of a 
plurality of search queries (col. 3 line 23-25 and col. 7 line 67-col. 8 line 10 as well as 
figure 3 drawing reference 302 and col. 12 step 1 ; e.g. keywords parsed from a query); 

a module (210-214) configured to merge (col. 8 line 22-23; i.e. adding found 
images to the result set) the plurality of query results (col. 3 line 32-34 and figures 6-7) 
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into a merged query result (figs. 6-7), tlie merged query result (figs. 6-7) being 
associated witli the plurality of search queries (col. line 20-24 and figure 3 which shows 
keywords associated to images); 

a module configured (216) to determine a first ranking sequence (fig. 4 drawing 
reference 412) of the merged query result (col. 7 line 15-18figs. 6-7); 

a module (200) configured to present the merged query result to a user 
according to the first ranking sequence (412 and 416); 

a module (220) configured to identify an input signal (figure 5 drawing reference 
502) from the user (200) indicating an interest (col. 7 line 22-23) in a first piece of 
information (col. 7 line 22-23; i.e. an image a user finds relevant/not relevant) in the 
merged query result (figures 6-7); 

a module (222) configured to identify a search query from the plurality of search 
queries (col. 3 lines 41-48, col. 10 step 4) associated with the merged query result (figs. 
6-7), the identified search query (col. 3 lines 41-48, col. 10 step 4) being associated with 
a query result (figure 3 drawing reference 304 and col. 8 line 57; e.g. an association 
link) including the first piece of information (col. 6 line 8-19), the query result from 
among the plurality of query results (col. 10 step 4; "for each positive example, check if 
any query is linked to it..." ; 

a module (222) configured to adjust (col. 3 line 49-50) a query factor associated 
with the identified search query (col. 5 line 24-29; e.g. a weight indicating a relevance of 
a keyword to an image) responsive to the input signal (col. 8 line 52-64); 
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a module (210-214) configured to locate a second piece of information in the 
query result (col. 9 line 13-19;"the feature and semantic matcher 152 also try to locate 
images that have similar low-level features as the example image selected by the user"; 
herein, the semantic and low-level features can be interpreted as the "second piece of 
information") associated with the identified search query (col. 3 lines 41-48, col. 10 step 
4); 

a module (220-222) configured to determine a score for the second piece of 
information (col. 9 line 16-19) based at least in part on the query factor col. 5 line 24-29; 
e.g. a weight indicating a relevance of a keyword to an image) associated with the 
identified search query (col. 3 lines 41-48, col. 10 step 4); 

a module (216) configured to determine a second ranking sequence (col. 12 step 
7) of the merged query result based at least in part on the score (col. 7 line 49-50 and 
figure 5); and 

a module (200) configured to present the merged query result to the user 
according to the second ranking sequence (col. 12 step 8 and drawing references 
508m, 518 as well as figures 6-7).. 

With respect to claim 26, Liu teaches the query system of claim 25, further 
comprising: 

a module configured to receive a user input (200); and 

a module configured to generate the plurality of search queries based on the 
user input (202). 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
Invention was made to a person having ordinary skill In the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner In which the Invention was made. 

Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Liu as 
applied to claim 1 above in view of Barrett et al. ("Barrett" hereafter) U.S. Patent 
Application 2003/0135490. 

With respect to claim 3, Although Liu teaches a lacl< of selection from a user of a 

first piece of information (i.e. the "No" branches in figure 5), they appear to lack teaching 
(with emphasis) the method of claim 1, wherein the input signal comprises lack of 
selection of the first piece of information for at least a specified amount of time from 
when the first piece of information is displayed to the user. 

Barrett, however, teaches wherein the input signal comprises lack of selection of 
the first piece of information for at least a specified amount of time from when the first 
piece of information is displayed to the user (paragraph 0012 step 16) for indicating a 
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time a user spends witli a result (i.e. duration) to calculate a user's interest for scoring 
purposes. 

Accordingly, in the same field of endeavor (i.e. information search and ranking), it 
would have been obvious to one of ordinary skill in the data processing art at the time of 
the present invention to combine the teachings of the cited references because the 
interest gauged by Barrett would have given Liu further feedback information for 
calculating relevance for the benefit of presenting optimized and refined results. 

Claims 13, 14, 19, 20, 27, and 28 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Liu as applied to claims 1,18, and 25, respectively, above. 

With respect to claim 13 and similar claims 19 and 27, although Liu teaches 
refining search results due to a user input (figure 5 drawing references 502, 504 and 
518 as well as col. 12 steps 3 and 7-8) to suggest refreshing a display, they do not 
expressly recite the method of claim 1 , further comprising increasing a refresh rate of a 
display of the merged query result to the user responsive to receiving input signals at an 
increasing frequency. 

However, it would have been obvious for Liu to teach increasing a refresh rate of 
a display of the merged query result responsive to receiving input signals at an 
increasing frequency because in refining results based on user interest, the rate of 
display would be increased due to user interest (i.e. in a situation of a user continuing to 
select "similar" and thus retrieving refined results). Such is apparent in Liu's browsing 
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behavior of a user (col. 9 line 53-59). Therein, it would have been obvious that a user's 
browsing behavior would determine subsequent queries for images so that refined (and 
thus refreshed) results would be presented In the next screen. Such a teaching of 
varying the refresh (refinement) rate would have been beneficial for the user to quickly 
refine results and locate the best result suited for their quer(y/ies). 

Claims 19 and 27 are substantially rejected under the same rationale as claim 13 
for reciting essentially the same subject matter. 

With respect to claim 14 and similar claims 20 and 28, Liu teaches he method of 
claim 1 , wherein the Input signal Is a first Input signal and the Interest is a first interest( 
drawing reference 502), further comprising: 

receiving a second Input signal Indicating a second Interest In a third piece of 
information (col. 12 steps 3 and 8 wherein step 8 repeats step 3 to suggest collecting a 
second indication of interest). 

Liu does not appear to expressly recite varying a refresh rate of a display of the 
merged query result to the user based at least In part on the duration between receiving 
the first input signal and the second input signal. 

However, it would have been obvious for Liu to teach varying a refresh rate of a 
display of the merged query result to the user based at least In part on the duration 
between receiving the first Input signal and the second Input signal because, depending 
on the user's Interest and feedback (I.e. a duration of Input signals Indicating If the user 
Is Interested or not), the results would have been refined (and thus refreshed) 



Application/Control Number: 1 0/81 5,074 Page 1 8 

Art Unit: 2167 

accordingly. Such is apparent when Liu describes a user's browsing behavior to select 
similar images and thereafter present refined (refreshed) results. Such a teaching of 
varying the refresh (refinement) rate would have been beneficial for the user to quickly 
refine results and locate the best result suited for their quer(y/ies). 

Claims 20 and 28 are substantially rejected under the same rationale as claim 13 
for reciting essentially the same subject matter. 

Response to Arguments 

Applicant's arguments with respect to the pending claims have been considered 
but are moot in view of the new ground(s) of rejection in view of Liu and Liu in view of 
Barrett. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Robert M. Timblin whose telephone number is 571-272- 
5627. The examiner can normally be reached on M-Th 8:00-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John R. Cottingham can be reached on 571-272-7079. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
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published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/ROBERT TIMBLIN/ 
Examiner, Art Unit 2167 

/John R. Cottingham/ 

Supervisory Patent Examiner, Art Unit 2167 



